WITNO06470100
WITN06470100
Witness Name: Richard Coleman
Statement No.: WITN06470100
Dated: 16 March 2023
POST OFFICE HORIZON IT INQUIRY
FIRST WITNESS STATEMENT OF RICHARD COLEMAN
1, RICHARD COLEMAN, will say as follows...
INTRODUCTION
1. lam currently a Minister of Religion within the Church of England. I have been
in this role since May 2011.
2. This witness statement is made to assist the Post Office Horizon IT Inquiry
(the "Inquiry") with the matters set out in the Rule 9 Request dated 17th
January 2023 (the "Request").
3. The Request relates to matters that took place more than 15 years ago.
Where I was involved in these matters, I have tried to recall events to the best
of my ability, but my recollection is very limited.
Page 1 of 11
WITNO06470100
WITN06470100
BACKGROUND
4. From April 1990 to June 1998 I worked for ICL in Cardiff as a hardware
engineer. In June 1998 I transferred to the Fujitsu 3rd line software support
team ("SSC") in Bracknell. In September 2005 I left Fujitsu to train to become
a Minister of Religion within the Church of England.
5. During my time in the SSC I was one of a number of support personnel
working under Mik Peach, the SSC Manager. It was part of our role to try and
provide solutions to issues that arose to the development team for them to
validate and then supply the actual fix. This also involved on occasion working
with the testing team to track down bugs as well as validating fixes from
development. I don't recall any interactions personally with the development
team, and whilst I don't recall any specific incidents, I do recall working with
the testing team on occasion to resolve a problem.
6. Mik gave us particular areas to focus on. Mine were the back end databases
in the datacentres at Bootle and Wigan that were used to configure Post
Office counters. Another large part of my role was in the building and
configuring of workstations for new staff coming into the SSC, as well as
working with Steve Parker to maintain and develop our internal website which
was used as a central resource by other teams within Pathway, which
included the Known Errors Log ("KEL") database, as well as other documents.
Page 2 of 11
WITNO06470100
WITN06470100
KEL SYSTEM
7. The Known Errors Log ("KEL") database was a collection of HTML documents
stored on the SSC internal website, and was the means by which information
on work arounds and solutions to issues were provided to 1st and 2nd line
support. They were also used to specify the evidence we needed in order to
investigate issues further should they crop up again.
8. I believe KELs had 4 major sections: What the customer reported, an
explanation of the problem, the solution to the problem, and any evidence that
we required to help us resolve the problem. There would also have been
options to add tags to the KELs, for example for a particular product or
software release.
9. To search the KEL database the support team would connect to the SSC
website and select the "Search the KEL database" link which would then
display a page where they could type in any words to be searched for. There
would also be a number of options that could be selected to allow easy
targeting of searches to, for example a particular product or release, in order
to help the support teams find the information they needed.
Page 3 of 11
WITNO06470100
WITN06470100
PINICL AND PEAK SYSTEMS
10.PinICL was the system used by the support and development teams to record
and manage calls. I do not recall any issues with it that prevented me from
doing my job, but there clearly were issues with it as PEAK was developed to
replace it, and I recall John Simpkins being heavily involved in_ its
development. I don't recall ever using it and so cannot comment on it.
1
.Before her retirement, Barbara Longley was the PinlCL administrator for most
of my time in the SSC, and she had a list of products and abilities for each
member of the SSC and she would assign calls as per their skill set and
workload. I do not recall who took over from Barbara when she retired, nor do
I recall how PinlCLs were prioritised.
12. The closing of calls would depend on the reason why the call had been raised
in the first place and whether that call needed to be passed on to another
team. For example, where a bug had been identified then I believe calls would
only be closed once a solution had been identified and the fix had been tested
and made available. Sometimes duplicate PinICLs would be raised for the
same problem and they would be closed as duplicates. There were a range of
codes used when closing calls to, I assume, enable management to gather
statistics on the types of calls that were being raised and why they were
closed. I recall that there was at least one team, though I do not recall the
name of the team, where we had to close the call on PinICL with a particular
response code for it to be passed on to them.
Page 4 of 11
WITNO06470100
WITN06470100
13.1 was a member of the SSC from June 1998 to September 2005 working with,
I believe, about 20 to 30 colleagues.
14. The role and purpose of the SSC was to handle software issues that could not
be handled by 1st or 2nd line support. As we had access to the source code
when we identified a bug we were also expected, if possible, to provide a
potential fix to the development team who would review the code and then
supply the actual fix that would be released to the counters. We also worked
with the testing team using their test rigs to investigate issues and at times to
assist them in testing fixes provided by development. I do not recall what
other teams there were within the support services structure.
15.Mik Peach was the SSC Manager. Steve Parker, John Simpkins and Pat
Carroll were the senior engineers who had worked on the project for many
years and their knowledge of the various systems was invaluable.
16.1 was not involved in the recruitment of personnel and so can not comment on
what qualifications or experience Mik required when looking to appoint
people, but the majority were experienced developers.
17.1 recall going on a Visual Basic training course, but I do not recall what other
training I received. We were expected to read the documentation provided by
Page 5 of 11
WITNO06470100
WITN06470100
development and by other SSC members for the various products that we
were tasked with learning. We were also required to train others, that Mik
identified, in our own areas of responsibility so that knowledge was shared
around the SSC.
18.My own area of expertise was the configuration databases in the data centres,
ACDB and OCMS were two of them, if I remember correctly.
19.As far as I can recall we were expected to carry out our work with due
diligence and where possible to correct the issue and to close calls accurately
with the correct response code. There were times when an issue would cause
a red alert and there would of course be pressure to meet the SLA. However if
we were unable to meet the SLA, which may involve a financial hit on Fujitsu,
Mik always handled that pressure from senior management so that we could
focus on getting the incident resolved, and to accurately document the issue
and its resolution both in the PinICL and in a KEL.
20.As far as I can recall the SSC had a good working relationship with all the
other support teams, and I don't recall there being any issues with SMC and
the volume of calls raised.
IDENTIFICATION AND RECTIFICATION OF BUGS, ERRORS OR DEFECTS
21.A number of the bugs listed in Appendix 2 of Bates and others v Post Office
Limited (No. 6) "Horizon Issues" [2019] EWHC 3408 (QB) that I have been
Page 6 of 11
WITNO06470100
WITN06470100
asked to comment on relate to Horizon Online of which I have no experience
as I only worked on what has come to be known as Legacy Horizon and so I
am unable to comment on those bugs. This leaves bugs 2, 6, 9, 10, 11, 12,
15, 18, 23, 24, 26, and 27.
22.Having reviewed the Technical Appendix supplied to me which goes into
detail on these bugs, whilst I recognise the majority of bugs I do not believe
they were ones that I investigated, and so can not comment on that, or if or
when they were reported to the Post Office, SPMs, etc. I do not recall being
involved in the investigation of calls to do with the branch accounts as there
were others, such as Anne Chambers and John Simpkins, who tended to
handle those types of calls. But once a solution had been identified I would
expect to have been allocated one or more calls and would simply have
followed the solution or work around as recorded in the relevant KEL. For the
bugs listed above I do not recall what those solutions were or how they were
to be actioned, or indeed whether I actioned any calls on them.
23.As far as I can recall, when a bug was identified the SSC would look to
provide a solution to the development team. If it was not possible for the SSC
to posit a solution the call may still be sent over to development with all the
evidence that we had collected for them to provide a solution, or the call
would be closed and a KEL created detailing the evidence we would require
to investigate further. Any solution from development would then be passed to
the testing team who would then test that it did actually fix the bug as
recorded in the PinICL. Once the fix had passed the testing team it would then
Page 7 of 11
WITNO06470100
WITN06470100
be distributed to the Post Office counters. I do not recall the mechanism by
which that was done.
24.Whilst we might suggest a possible work around, I can not comment on why a
work around would be used instead of implementing a fix as those sorts of
decisions were not taken by the SSC. As far as I can recall, work arounds
would only be implemented with the agreement of development and the Post
Office, and I would expect that agreement to be recorded in the PinICL and
KEL.
25.1 don't recall any procedures or policies in place restricting what we could tell
SPMs in relation to bugs, nor do I recall any pressure placed upon me or my
colleagues not to pass information to SPMs or the Post Office. Whilst I did
speak to SPMs I do not recall any of those conversations. We were expected
to fully record our findings in PinICL. How or when, or indeed if, that
information would be passed on to the Post Office would be for management
to decide.
26.As already mentioned, although we were aware of various SLAs, and we were
expected to do our best to meet those SLAs, I believe it was more important
to Mik Peach, as SSC Manager, that we identify the cause of the error and
provide a solution than it was to meet any particular SLA.
27.1 do not recall investigating any problems with Riposte itself, and as far as I'm
aware they would have been investigated as any other problem and any
Page 8 of 11
WITNO06470100
WITN06470100
potential solution developed by the SSC would have been passed on to
development in the usual way for them to raise the issue with Escher.
28.Again I do not recall there being any pressure placed on us to avoid finding
bugs, errors or defects. When a call was passed to us we were expected to
investigate and resolve the issue and to record our findings in the PinlICL and
KEL as appropriate.
REMOTE ACCESS
29.Normally we would access the Riposte messagestore on the Correspondence
Servers and we had workstations specifically for that which were on a
completely separate network to the workstations we used to access PinICL
and email, etc. Access to the Correspondence Servers would either be via a
Windows Command window or through one of the tools developed internally
by the SSC.
30.1 do not recall remote access onto a Post Office counter being a common
function as the Riposte data we would need to look at for most problems
would normally be replicated to the Correspondence Servers, which we would
access. However, I believe there would be times when we would need to
access files directly on the counter which were not replicated to the
Correspondence Servers, such as the Windows Event logs, which we would
need to look at if we suspected a hardware issue for example.
Page 9 of 11
WITNO06470100
WITN06470100
31.Physical access into the SSC was restricted with card keys that were
separate from the rest of the building so only the SSC had access to the floor
we were on. I do not recall what other security measures were in place or
what audit data was automatically captured when using remote access.
32.In terms of editing or deleting data, the design of the Riposte messagestore
was such that we could not edit or delete individual items of data within the
messagestore. Any such attempt would have caused Riposte to stop working
and to generate alerts. What we could do was to delete the entire
messagestore on a counter and then allow Riposte to rebuild it from the
transactions that would be stored on the other counters within the Post Office.
We could insert new transactions into the messagestore via the
Correspondence Servers which would then replicate to the counters, but
would only do so when authorised by the Post Office, and it would be
recorded that we were doing that in the PinICL. I believe we put the PinICL
call reference into the messagestore messages that we inserted. I do not
recall if, or how, that would be relayed to the SPM.
GENERAL
33.1 can not comment on the training that SPMs received in the use of the
system as I was not privy to that information. Nor was I aware of what advice
was being given by 1st and 2nd line support. And given the passage of time I
do not recall specific incidents to enable me to comment on what changes
Page 10 of 11
WITNO06470100
WITN06470100
would have helped improve the advice and assistance received by SPMs or
what the causes of those problems were.
Statement of Truth
I believe the content of this statement to be true.
Dated: 16 March 2023
Page 11 of 11